Questioning the Effectiveness of Contextual Online Help: Some Alternative Propositions

Most online help systems on the market prove unsuited to the actual needs of novice users. Although recent research has focused on the design and implementation of contextual, adaptable or adaptive, help systems, the actual effectiveness and usability of such systems has not yet been assessed extensively. We present an experimental ergonomic evaluation of a contextual help system implementing human expert strategies. 18 potential users interacted with a standard text processor, using two help systems, a contextual one versus a non contextual one. Each system was simulated using the Wizard of Oz paradigm. The contextual help strategies were defined from the analysis of novice-expert dialogues. Contextual help proved more effective than non contextual help as regards successful task execution, this result being statistically significant. Concerning the influence of individual cognitive characteristics on the subjects' performance, analyses based on psychological tests suggest that contextual help facilitates message understanding, whereas it is less effective than standard help for subjects with low exploratory capabilities.